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Preface 

This report provides information on the preparation provided to teacher candidates from teacher training programs 
so that they can fully use assessment data to improve classroom instruction. It follows a similar shorter report we 
released in March 2012. 1 In this newest report, we have significantly expanded our initial sampling to encompass 180 
undergraduate and graduate programs housed in 98 institutions in 30 states. 2 

In early 2013, with U.S. News &World Report as the publisher, we will release a comprehensive review of all teacher 
preparation programs in the United States. That review will also include further analysis of teachers’ preparation in use 
of assessment for instructional improvement. 

Our purpose with both preliminary reports is to encourage teacher preparation programs, along with the federal and 
state agencies supporting and overseeing these programs, to focus more on the importance of future teachers’ knowledge 
and skills in the increasingly critical area of assessment. 



1 http:/ /www.nctq.org/p/edschools/docs/assessment_publication.pdf 

2 Institutions included in the sample are listed in Appendix 1 . Of the 1 8 0 programs, 8 7 (48 percent) are elementary (84 percent 
undergraduate and 16 percent graduate) and 93 (52 percent) are secondary (74 percent undergraduate and 26 percent 
graduate) . 
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Introduction 

Why teacher knowledge of assessment is critical 

Judging from some of the debates on the use of data in schools, one might think that the only reason for prospective 
teachers to learn how to interpret student performance data is to interpret the standardized tests that are mandated by 
states and the federal government. 3 The imperative for teachers to be able to use data should not, however, be limited to 
such tests. Data from assessments of all kinds have always been central to good instructional practice. Effective instruction 
requires that teachers have a well grounded sense of student proficiency in order to make a daunting number of 
instructional decisions, such as making snap judgments in the midst of interactions with students, and planning lessons, 
be they for the next day, the next unit or the entire school year. Setting aside many of the 1,200 or so instructional 
decisions teachers make on average each day (“Has this student sufficiently mastered the material so that he can work 
independently until recess?” “What question can I pose that will get the guys in the back row to perk up?”), the number 
of instructionally significant decisions made daily is about 60 (“Which topics should I include on the unit test?”). 4 

Teachers have always needed a firm grasp of their students’ understanding to make appropriate instructional decisions. 
Many of the first signals attuned teachers pick up on are not the result of any assessment. A teacher needs to be alert 
to the less tangible indications that students are not grasping new material, from a furrowed brow to “acting out.” That 
knowledge must be combined with more tangible information gleaned from assessments in the form of spontaneous 
questioning, homework assignments, quizzes, tests or state-mandated assessments. From informal oral checks to 
computer-administered tests, assessment provides the basis for instructional decisions, making it critical even in 


3 This report does not touch on the need for teachers to understand the use of student performance data in evaluating their own 
performance, but that understanding is also critical. 

4 Hosp, J. L (December 2010). Linking assessment and instruction: Teacher preparation and professional development. 

A TQ connection issue paper on improving student outcomes in general and special education. Washington, DC: National 
Comprehensive Center for Teacher Quality. (Access at: http:/ /www. tqsource.org/pdfs/TQ_IssuePaper_AssessInstruct.pdf) 
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tiie absence of “high stakes” standardized tests. * * * 5 Further, the use of frequent assessments is a 
hallmark of the teaching profession in countries whose students outperform our own. 6 For 
example, Finland — cited often for a variety of educational features, among them a dearth 
of standardized testing — trains teachers to use classroom assessment with a high level of 
sophistication. 7 Moreover, several recent studies suggest that assessment itself appears to enhance 
learning through strengthened memory representation. 8 


“Data driven 
instruction” 
represents a 
more organized 
and collaborative 
commitment to use 
data from a variety 
of assessments and 
other sources in 
order to develop or 
adjust instruction. 


The new focus on " data driven instruction" 

While any teacher who casually peruses quiz results and then decides to re-teach a concept 
could be described as “using data to drive instruction,” that phrase has taken on new meaning 
in this era of increased school accountability. “Data driven instruction” represents a more 
organized and collaborative commitment to use data from a variety of assessments — as well 
as information on student attendance, student engagement, demographics, attendance and 
school climate — in order to develop or adjust instruction. 

Districts may vary in their organizational commitment to data driven instruction, but it is 
clear that teachers increasingly find themselves not just working in isolation to divine the 
instructional implications of assessment results, but also working collaboratively with colleagues 
to use results to improve the performance of individuals, classes, grades or the entire school. 
For instance, the state of Delaware recently threatened to withhold $2.5 million from a 
school district because the district had not properly implemented a 90-minute common 

planning time in which teachers “take time to strategize and learn to use the voluminous 

data produced by the state’s new computer assessments to help students in areas where they 

are struggling most.” 9 


5 The stakes of high stakes assessment were recently raised by research connecting high standardized 
tests scores with a wide variety of measurable positive outcomes in adult life: Chetty, R., Friedman, 
J. N., & Rockoff, J. E. (201 l).The long-term impacts of teachers: Teacher value-added and student 
outcomes in adulthood (Working Paper 17699). Cambridge, MA: National Bureau of Economic 
Research. (Access at: http://www.nber.org/papers/wl7699) 

6 Tucker, M. S. (May 24, 201 1). Standing on the shoulders of giants: An American agenda for 
education reform. Washington, DC: National Center on Education and the Economy. (Access at: 
http:/ /www.ncee.org/wp-content/uploads/20 1 1 /05/Standing-on-the-Shoulders-of-Giants-An- 
American-Agenda-for-Education-Reform.pdf) 

7 Sahlberg, E (2011). Finnish lessons: What can the world learn from educational change in Finland? 
New York: Teachers College Press. See p. 66 for a discussion of assessment. 

8 Chang, C.-Y.,Yeh,T-K., & Barufaldi, J. R (20 10). The positive and negative effects of science concept 
tests on student conceptual understanding. International Journal of Science Education, 32(2), 265- 
282; Karpicke, J. D., & Blunt, J. R. (201 1). Retrieval practice produces more learning than elaborative 
studying with concept mapping. Science, 33 1 , 772-775; McDaniel, M. A., Agarwal, P. K., Huelser, 
B. J., McDermott, K. B., & Rodeiger III, H. L. (201 1 ). Test-enhanced learning in a middle school 
science classroom: The effects of quiz frequency and placement. Journal of Educational Psychology, 
103(2), 399-414; Rohrer, D., & Pashler, H. (2010). Recent research on human learning challenges 
conventional instructional strategies. Education Researcher, 39(5), 406-412; Rohrer, D., Taylor, K., & 
Sholar, B. (2010) . Journal of Experimental Psychology, Learning, Memory, and Cognition, 36(1), 
233-239. 

9 Dobo, N. (December 16, 2011). Delaware tells Brandywine schools to make time for teachers 
to plan. Delawareonline.com (Access at: http://www.delawareonline.com/article/201 11216/ 
NEWS/ 1 121 60335/Delaware-tells-Brandywine-schools-make-time-teachers-plan) 


2 


INTRODUCTION 


The school districts in the nation that do best in the face of the challenge of educating 
disadvantaged students have become obsessive about using data to drive instruction. Two 
of the last three winners of the highly coveted Broad Prize for Urban Education, an award 
to honor “urban school districts that demonstrate the greatest overall performance and 
improvement in student achievement while reducing achievement gaps among low-income 
and minority students,” each demonstrated a relentless focus on data . 10 

The 2011 Broad Prize winner, Charlotte-Mecklenburg Schools (CMS) in North Carolina, 
narrowed achievement gaps between African-American and white students in reading and 
math at all school levels. The district's intense approach to data-based decision making is one 
factor in this success: All educators in CMS are trained in Data Wise, a structured system to 
improve instructional and organizational practices based on data, and every school has a data 
team. Teachers, administrators and coaches can access a wide variety of learning data on the 
district's online portal, which gives them the capacity to meaningfully track student performance 
and adapt quickly to learning difficulties. 

Aldine Independent School District (Texas) won the Broad Prize in 2009, with a strong 
record of academic success for its students and a student performance data initiative that 
parallels CMS's. The district developed its own online curriculum and assessment database 
(in part fed by data from frequent common student assessments), which provides teachers, 
instructional leaders and district staff with valuable same-day, aggregated and disaggregated 
student achievement information. This data empowers them to monitor and modify instruction 
until students demonstrate that they have mastered a subject. District staff can use the data 
to identify students who need special help, such as students with disabilities or English language 
learners, and can then use this information to implement targeted staff development in schools 
where problems are detected. Individual professional development plans are also shaped by 
test data. 


It is fair to say 
that the school 
districts in the 
nation that do 
best in the face 
of the challenge 
of educating 
disadvantaged 
students have 
become obsessive 
about using data 
to drive instruction. 


Ideally, the training that every aspiring teacher receives would fully prepare them for the 
growing demands of data driven instruction, employing all potential sources of data that could 
bear on the academic performance considered, including attendance records, school climate 
issues such as suspension rates, the nature of needs posed by a variety of English language 
learners and so on. This report, however, focuses more narrowly on the training teacher 
candidates receive only in using data derived from student assessments — ranging from 
classwork practice to state tests — to improve instruction. All references to “data” refer to 
data from such assessments. References to coursework that addresses assessment and assessment 
data will be referred to as “assessment coursework.” 


1 0 http:/ 7www.broadprize.org/ past_winners/20 1 1 .html 
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The evidence 
for the connection 
between using 
data to drive 
instruction 
and student 
performance 
is emerging, just 
as the practice 
of using data is 
emerging. 


Previous research 

There has been little research on the efficacy of teacher training in assessment. A search of 
peer-reviewed journals over the last decade produced 26 studies, the vast majority of which 
do not meet common standards for research. These studies generally have a very small number 
of subjects (often the researcher’s own students) and make no attempt to examine the effects 
of any “treatment” on teacher effectiveness. The core finding of these weak studies is that 
teachers have difficulty analyzing data from classroom assessments and are therefore unable to 
use the data to guide instruction. The small number that consider the effects of assessment 
coursework on teacher effectiveness attempt to do so without using data from externally 
validated data sources. (A list and categorization of these studies is found in Appendix 3.) 

The evidence for the connection between using data to drive instruction and student 
performance (sometimes called “an inquiry cycle”) is emerging, just as the practice of using data 
is emerging. * 11 At this early stage, however, and using strict definitions of adequate research, an 
Institute on Education Sciences (IES) report found “no research that assessed the impact on student 
achievement of using an inquiry cycle, or individual steps within that cycle, as a framework for data 
analysis. . . Tire panel determined that the level of evidence to support [the recommendation for its 
use] is low.” Notwithstanding this conclusion, however, IES did publish a lengthy “practice 
guide” on the topic, implicitly recommending that educators implement well-designed 
inquiry cycles. 12 

Since publication of the IES overview, one study has found that a data driven reform initiative in 

500 schools located in seven states led to district- wide improvements in student mathematics 
achievement, but no such improvements in reading achievement. 13 


1 1 Black and Wiliam reviewed 250 studies of formative assessment and claim that it has a large potential 
impact, to arrive at this conclusion, but since these studies are older (they were conducted between 
1988 and 1 998) when most education research had weaker designs than they do today, an untold 
number may be suspect. Only the fact that there was a consistent finding for such a very large 
number of studies lends credence to this conclusion. Black, E, & Wiliam, D. (1998). Assessment and 
classroom learning. Assessment in Education, 5, 7-74. 

1 2 National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences. 
(September 2009) . IES practice guide: Using student achievement data to support Instructional 
Decision Making. Washington, DC: U.S. Department of Education. (Access at: http:/ /ies. ed.gov/ 
ncee/ wwc/pdf / practice_guides / dddm_pg_0 92909 .pdf) 

1 3 Carlson, D. , Borman, G. , & Robinson, M. (20 1 2) . A multistate district-level cluster randomized 
trial of the impact of data-driven reform on reading and mathematics achievement. Educational 
Evaluation and Policy Analysis, 33(3), 378-398. A soon-to-be-released study by the Council of the 
Great City Schools found modest improvements in middle school mathematics and elementary 
reading. 
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A professional imperative from “day one" 


U.S. Secretary of Education Arne Duncan made data-driven decision making a national priority, 
requiring states to improve their data systems and create high-quality assessments to be eligible 
for a Race to the Top grant. Federal and state governments have devoted increasingly significant 
resources to the data-driven decision making reform effort. Since 2005, over $500 million 
of federal funding has been spent through the Statewide Longitudinal Data System (SLDS) 
program for development of states’ technology infrastructures. 14 

Fortunately, progress has been made. The Data Quality Campaign (DQC) released a report in 
December 2011 indicating that every state now has the ability to match individual students’ 
test records from year to year in order to measure academic growth. 15 

Districts, too, have been making significant investments. For example, Aldine Independent 
School District, the 2009 Broad Prize winner, invested $930,000 over a six-year period in 
its online curriculum and assessment database — a significant sum given that the district has 
only 64,000 students. 16 

While the data produced by district and state systems may have value in terms of accountability, 
the full educational potential of these systems can only be realized if teachers have the capacity 
and motivation to use the data they make available. Yet anecdotal reports indicate that teachers 
are not taking full advantage of the data that assessments yield. 17 Even among teachers that 
consider themselves heavy users of student performance data, less than half see data that 
originate outside of their own classroom (such as data from district or state-required tests) 
as very important. Among teachers who report being light users of data, the proportion of 
those who see such external data as important is fewer than one in four. 18 


Now, every student 
comes with instructions. 




STAR 

n Enterprise 

an nwractM* *our at 
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A November 201 1 advertisement 
in Education Week. The tools for 
monitoring and reporting on student 
performance are an increasingly 
important feature of K-l 2 education. 


1 4 National Center for Education Statistics. (2010). Statewide longitudinal data systems grant program. 
(Access at: http://nces.ed.gov/programs/slds/) 

1 5 Sparks, S. (20 1 1 , December 7). Survey Shows Nearly All States Can Track Data on Students. Education 
Week. While we agree that much progress has been made by states, our 2011 State Teacher Policy 
Yearbook (http://www.nctq.org/stpyl 1/reports/stpyl l_california_report.pdf) discusses deficiencies 
in California’s capacities. 

1 6 According to Ben Wilson, Aldine ’s Assistant Superintendent of Community and Governmental 
Relations, approximately 70 percent of these expenditures were funded by the district, with the 
remaining 30 percent funded by grants. 

1 7 Of course, the responsibility for poor use of data and data systems is shared between teachers and 
their districts when the district’s system is poorly designed. For example, New York City spent more 
than $80 million on data system design and development, but an audit revealed that less than half 
of educators accessed the system in spring 2011. (Access at: http://www.comptroller.nyc.gov/ 

bureaus/audit/audits_201 2/ 1-23-1 2 71 1 1-1 18A.shtm) An online blog reports that New York 

City teachers complain that the system offers them too litde information, and parents say it’s hard 
to access. The fact that some teachers and administrators do want data is revealed by the fact that 
enterprising teachers and schools have created alternative systems for their own use that they are 
also selling to other public schools. (Access at: http://gothamschools.org/2010/09/ 15/frustrated- 
with-citys-data-system-teachers-build-their-own) 

18 Primary Sources: America’s Teachers on America’s Schools: A Project of Scholastic and The Bill and 
Melinda Gates Foundation (2010). (Access at: http://www.scholastic.com/primarysources/ 
pdfs/Scholastic_Gates_03 10. pdf) The most recent report in this series issued in 2012 does 
not change these conclusions. (Access at http://www.scholastic.com/primarysources/pdfs/ 
Gates20 1 2_full.pdf) 
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“In my experience, 
when teachers are 
able to see first- 
hand how the use 
of data can impact 
the achievement of 
their students they 
can be convinced 
of the value of data 
driven instruction.’’ 

— Abbey Goldstein 
Manager, Strategic Initiatives 
District of Columbia 
Public Schools 


Ironically, many teachers seem to see little use for external assessments even considering the 
purposes for which they are clearly designed: determining whether students have a deep 
understanding of content and deciding whether to move them to the next grade. In a recent 
survey, no more than three in 10 teachers ranked these tests as valuable for such purposes. 19 

The Washington Post recently printed some reactions by teachers to a column discussing the 
growing paperwork burden placed on public school teachers. These responses revealed both 
the ubiquity of “data crunching” and the overwhelmingly negative sentiments about it: 

■ It’s all about data these days, and our professional development days are all about 
crunching numbers, looking at data and sifting through scores. NEVER is there any talk 
about how to create engaging and effective and interesting lessons. 

• Lost in all of this insanity of being driven by the data is the teacher’s cry in the wilderness: 
Where is the responsibility for the student to learn ? 
m I would argue that in order to improve student achievement, teachers need time to actually 
instruct children. Instead, too much time has been spent this last quarter on ... assessing 
students. 20 

School district personnel do report that these attitudes can shift with evidence of the positive 
impact on student learning of data driven instruction. 

Identifying a core for assessment knowledge 

School districts, states and teacher preparation training programs have yet to establish what 
knowledge a new teacher should have in order to enter a classroom with some facility for 
applying data to improve classroom instruction. In fact, the field has struggled to incorporate 
data-driven decision making into its program sequences. As researchers Ellen Mandinach and 
Edith Gummer wrote: 

The held of data-driven decision making falls between the cracks of traditional courses. 
It is not introductory statistics or measurement; it is not instruction, pedagogy, or methods. 
Data-driven decision making lies at the intersection of those courses. 

— The Complexities of Integrating Data-Driven Decision Making into Professional 
Preparation in Schools of Education: It’s Harder Than You Think 11 

In 2010, NCATE, the largest accreditor of teacher preparation programs, published a report 
on assessment in teacher preparation 22 with the general recommendation that “candidates 


19 Survey results can be accessed at http:/ /www.nwea.org/every-child-multiple-measures 

20 What’s lost in the paperwork. (201 1 , November 21). The Washington Post, p. B2. 

2 1 Mandinach, E. B., & Gummer, E. S. (2011, May 1 2) . The complexities of integrating data-driven 
decision making into professional preparation in schools of education: It’s harder than you think. 
Report from an Invitational Meeting Presented to the Spencer Foundation. (Access at: http:// 
educationnorthwest.org/webfm_send/ 1133) 

22 Assessment as a critical element in clinical experiences for teacher preparation. (Access at: 
http:/ / www.ncate.org/LinkClick. aspx?fileticket=oo50CSYDEFM%3D&tabid=7 1 5) 

6 | National Council on Teacher Quality 


INTRODUCTION 


be presented with multiple and rich course material in their preparation that will enable 
them to become assessment-literate and data-wise.” It did not however, provide specific 
suggestions as to how such material could be integrated into existing coursework or provided 
through new coursework. 23 


After a review of the literature and the opinions of experts in the field, 24 we delineated the 
following three domains of knowledge needed by teacher candidates in the area of assessment 
preparation: 

1. How to measure student performance using assessments: “Assessment Literacy” 

2. How to analyze student performance data from such assessments: “Analytical Skills” 

3. How to use student performance data from assessments to plan instruction: 

“Instructional Decision Making” 

These three domains of knowledge formed the basis of our examination of institutional 
practice in assessment. As we sifted through the syllabi for professional coursework that 1 80 
programs require of their teacher candidates, we classified a course as relevant if it provided 
content associated with one or more of the three domains. We then proceeded to rate the 
comprehensiveness of a program’s coverage. 

A full discussion of each domain, the rubrics used for evaluation in each of the three domains, 
and descriptions of the relevant features of sample programs under each rating category are 
found in Appendix 2 . 


The three domains 
of assessment 
knowledge needed 
by teacher candidates: 

1. Assessment literacy 

2. Analytical Skills 

3. Instructional 
Decision Making 


23 Appendix 4 contains a full review of state regulations, institution accreditation standards and 
professional standards, examining the degree to which they have articulated the requisite teacher 
knowledge and performance. 

24 Notably Dr. John Hosp, Associate Professor, Department ofTeaching and Learning, University of 
Iowa; Dr. Barry Kaufman, President, BK Education Consulting Services; and Dr. Ellen Mandinach, 
Senior Research Scientist, WestEd. 
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Methodology 

The study sample 

The sample for this preliminary report is 180 teacher preparation programs for both elementary and secondary teacher 
candidates, housed in 98 institutions of higher education (IHE) in 30 states. These institutions were selected for the 
sample because they were in the first states responding to open records requests from NCTQ to provide syllabi associated 
with required professional coursework toward initial certification, an aspect of our National Review of Teacher Preparation 
Programs. 15 

The graphics that follow illustrate the characteristics of the sample relative to the total population of institutions offering 
teacher preparation programs, as well as illustrating the distribution of the sample relative to institutional selectivity 
in admissions as measured by average SAT scores of all students. 

While the sample is overwhelmingly composed of public institutions, we have no reason to believe that this biases 
results. The results from our analysis of assessment coursework in Illinois 26 suggest that there is little difference in the 
practices of public and private institutions. 27 


25 http:/ /www.nctq.org/p/edschools/home.jsp 

26 Greenberg, J., & Walsh, K. (2010). Ed school essentials: A review of Illinois teacher preparation. Washington, DC: National 
Council on Teacher Quality. (Access at: http://www.nctq.org/edschoolreports/Illinois) 

2 7 While the rating process in Illinois was not exactly the same as used here or as will be used in the National Review, programs in 
private institutions in Illinois had average ratings of 3.0 on a 4-point scale, whereas ratings of public institutions averaged 2.8. 
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Public and private institutions* 





Public Private 


■ All institutions (n= 1 370) 

■ Institutions in the sample (n=98) 

*Data from Title II reports 

Data were obtained mostly through 
open records requests to which only 
public institutions must respond. As a 
result, while just over one-third of the 
IHE's offering teacher preparation are 
public institutions, about 95 percent 
of the IHEs in the sample are public. 


National accreditation of institutions 



All Institutions 
institutions in this 
sample 


■ All institutions (n=1370) 

H Institutions in the sample (n=98) 

Because many states require that 
IHEs offering teacher preparation seek 
accreditation, a disproportionate number 
of accredited IHEs are public. This may 
explain why the proportion of accredited 
IHEs in the sample is much larger than in 
the proportion of accredited IHEs overall. 


Diversity of institutions* 


100 

90 



0-30% 31-50% 51-100% 

Percentage non-white students 

■ All institutions (n=1114) 

■ Institutions in the sample (n=97) 

*Data from Title II reports 

In terms of minority enrollment, the 
sample is representative of the population 
of IHEs offering teacher preparation. 


Number of teachers produced annually by institutions* 
« 50 



Very low Low Medium High Very high 

1-50 51-150 151-300 301-850 851+ 

■ All institutions (n=1334) 

■ Institutions in the sample (n=98) 

*Data from Title II reports 

The distribution of IHEs offering teacher preparation is dominated 
by institutions producing 50 or fewer graduates annually in all 
programs, many of them private. The fact that the sample contains a 
disproportionate share of public IHEs may explain why the sample has 
a more even distribution across four of five categories of production. 


Admissions selectivity of institutions in the sample* 
50 


on 

C 



701- 801- 901- 1001- 1101- 1201- 1301- 1401- 1501- 

800 900 1000 1100 1200 1300 1400 1500 1600 


Median SAT score (combined math and reading) 
for all enrolled students 
(National median SAT score is approximately 1011) 

* Data obtained from IPEDS for all but four IHEs for which scores were 
obtained from Cappex. 

Selectivity information based on average student SAT scores is not available 
for a large number of IHEs. This only illustrates the distribution of SAT scores 
for the sample. 
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The methodology for selection of coursework 

Any required course was considered relevant for further analysis if the title of the course or its description in the IHE’s 
catalog indicated that the topic of assessment might be a significant feature of instruction and/ or practice. 28 The syllabi 
for all such courses were reviewed. 

Some courses were not classified as relevant to the aims of this study even though their title and/ or description referenced 
the topic of assessment. These include courses in educational psychology that are taught in a psychology department 
(rather than an education department) and that are not designed for teacher audiences. These courses’ treatment of 
assessment would not be sufficiently contextualized to K- 1 2 schooling to include in this study. Courses in which literacy 
was the main topic were also excluded. 29 

For the purpose of evaluating programs preparing elementary and secondary teachers, all professional courses referencing 
assessment in the title or course description (with the exceptions noted above in educational psychology and literacy) 
were evaluated. For the purpose of evaluating programs preparing secondary teachers, one of four pathways for secondary 
certification was randomly selected (English, mathematics, science or social studies) and the methods course for the 
path chosen was also evaluated, regardless of whether assessment was mentioned in its title or description. 30 

After deciding which courses appeared to be relevant, programs were evaluated on the basis of the following information: 

■ The course syllabi stating the course objectives, lectures, assignments, textbooks and required readings; 31 

■ Information provided by publishers’ descriptions of the content of required textbooks; and 

■ Requirements for capstone projects (often called “teacher work samples” or “portfolios”). 32 


28 We had anticipated that institutions would assist in the identification of assessment coursework through a voluntary curriculum 
mapping process, but the lack of cooperation from institutions in the National Review sample made this impossible. Institutions’ 
requests for excessive payments for data also made it impossible to obtain all professional coursework syllabi for all schools 
to examine them for coverage of assessment. It is possible that courses treat assessment but the title or course description did 
not so indicate. To ascertain the potential for the coverage of assessment that is not revealed by title or course description, we 
conducted an analysis of coursework in 1 8 programs in the sample for which we had all syllabi available for evaluation. Our 
analysis indicates that titles and descriptions of many methods courses do not mention assessment, even though assessment is 
the topic of one or two lectures. However, with the exception of one course in one program, there were no practice assignments 
on assessment other than in the course of writing lesson plans, and there was no instruction or practice that would be 
evaluated in the second and third assessment domains that will shortly be described. Both because the program in which 
non-lesson planning-related practice was found already had adequate practice and because instruction and practice was related 
only to the first domain, had this instruction and practice been included in our evaluation, none of the additional coverage 
of assessment in these courses in the 1 8 programs would have changed the overall rating of any of the programs in which it 
was found. 

29 Assessment in literacy instruction (ranging from typical progress monitoring to the assessment that is integral to systematic 
programs such as “Response to Intervention”) requires a more specialized evaluation than provided here. Our evaluation 
of literacy coursework against our early reading standard addresses assessment and assessment strategies. For example, we 
evaluate whether instruction on assessment strategies focuses on systematic skill building as opposed to approaches such as 
running records. 

30 It is reasonable to assume that assessment-informed planning would be addressed in the separate subject-specific methods 
course for each pathway to certification. In other words, it is valid to evaluate only a sample of methods courses in a program 
in order to produce a general evaluation of methods coursework because it parallels the experiences of teacher candidates. 

3 1 Several researchers evaluating teacher preparation in literacy assessment-evaluated programs have used this same source of data: 
McCombes-Tolis, J., & Spear-Swerling, L. (201 l).The preparation of pre-service elementary educators in understanding and 
applying the terms, concepts, and practices associated with response to intervention in early reading contexts. Journal of 
School Leadership, 21(3), 360-389. 

32 Teacher Performance Assessments (TPAs) were also evaluated as capstone projects. For all projects, we accounted for any 
assessment-related project assignments, including whether these projects required that candidates submit artifacts from earlier 
coursework that reflected assessment assignments or entailed new assessment assignments. 
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A total of 455 
courses in 1 80 
programs were 
evaluated for 
their coverage 
of assessment. 


On this basis, a total of 455 courses were evaluated, anywhere from one to six relevant courses for 
each program. The average number of courses in each program that reference assessment is 2.5. 

The validity of using syllabi 

Analyses of syllabi have long been an accepted part of the evaluation of teacher preparation by 
state agencies, accrediting organizations and multiple research studies. The NCTQ methodology 
mirrors this approach, looking only for topics that are considered so essential that an instructor 
would not have failed to mention them on the syllabus if she or he intended to teach the 
topics at all. 33 Even in cases when syllabi are cryptic, careful examination yields valuable 
information about what teacher candidates are expected to learn and practice. For example, if 
a syllabus was unclear whether an assignment for an “assessment analysis” included analysis 
of both classroom and standardized assessments, it was assumed that both were addressed, 
provided that a lecture delivered before rather than after the assignment was due dealt with 
standardized tests. In the absence of clear lecture topics or practice assignments, course 
objectives were used to assess course content. 34 


33 Teacher educators often object to the use of syllabi to assess the core of instruction. Most objections 
seem to be based on the proposition that the most important concepts in a course can be so 
deeply embedded that they are invisible. In the words of one education school dean, if something is 
a “natural part” of a course it “need not be explicitly identified.” In fact, it is exactly the “natural 
part” of a course that is routinely expected by students, instructors, college administrators, trustees 
and accreditors to be described by syllabi that are the topics salient to this analysis. 

34 See “Sample program earning a ‘4’” on p. 37 for an example of coursework evaluated in the 
absence of clear lecture topics. 
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Findings 

How adequately does teacher preparation program coursework address 
Domain #1 , Assessment Literacy ? 

Assessment Literacy 

An understanding of the taxonomy of assessment and a familiarity with the different types of classrooms 
and standardized assessment. For example, teacher candidates should understand the distinction between 
formative and summative assessment and have developed several types of each form of assessment. 
Appendix 2 contains a more detailed description of this domain, the rubric used for evaluating programs in 
this domain area, and descriptions of the relevant features of sample programs in each of the rubric's rating 
categories. 


How adequately does coursework address "Assessment Literacy”? 



Virtually 

Very 

Limited 

Partially 

Adequate 

none 

limited 

Coverage 

adequate 



Undergraduate elementary (n=72) ■ Graduate elementary (n=14) 

■ Undergraduate secondary (n=69) ■ Graduate secondary (n=24) 


Only 2 7 percent of the programs in the sample cover literacy topics adequately, with an additional 2 7 percent 
doing so with partial adequacy. More than half of all programs have no, very limited or limited coverage. 
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In the first domain, only five programs (3 percent) in our sample earned the lowest rating, and one in five programs (2 1 
percent) earned the highest. Findings in this domain are clearly more positive than findings in the two other domains. 

Programs that earned the highest score provided: 

1. Comprehensive coverage of both classroom and standardized assessment (including key concepts such as validity 
and reliability) , and 

2. Practice on developing and scoring assessments. 

Most of the remaining programs provide teacher candidates with at least basic exposure to both formative and summative 
assessment. 35 Moreover, in about two-thirds of the programs (67 percent), the coursework contained at least some 
reference — albeit, in most cases, fairly minor references — to standardized testing. We also noted little explicit skepticism 
about state assessments and accountability systems, finding it in only six programs (3 percent). Given that a recent 
survey of teacher educators found that only 24 percent of teacher educators believe it “absolutely essential” to produce 
“teachers who understand how to work with the state’s standards, tests and accountability systems,” 36 this absence of 
explicit skepticism is commendable. 

How adequately does teacher preparation program coursework address 
Domain #2, Analytical Skills? 

Analytical Skills 

Understanding how to dissect, describe and display the data that emerges from assessments. For example, teacher candidates 
might learn how to disaggregate class results on a series of assessments in a unit to review and present differential subgroup 
performance in a table or chart. Appendix 2 contains a more detailed description of this domain, the rubric used for evaluating 
programs in this domain area, and descriptions of the relevant features of sample programs in each of the rubric's rating 
categories. 


3 5 Our findings run contrary to the results of a recent ETS study that found formative assessment mentioned explicitly in only 
three of the 22 programs (14 percent) that ETS examined. (Access at: http:/ /blogs.edweek.org/edweek/teacherbeat/201 1/12/ 
how_do_we_train_teachers_to_us.html) In an examination of the first 27 institutions (48 programs) evaluated, (approximately 
one-quarter of the total programs evaluated) , we found explicit references in roughly half of the programs (14/27 or 52 
percent). We found implicit references to assessment (e.g, a lecture on “how to determine what students know”) and/or 
capstone projects that required teachers to design formative assessments in all but one of the remaining 13 programs. 

36 Cracks in the ivory tower?: The views of education professors circa 2010. (Access at: http://educationnext.org/cracks- 
in-the-ivory-tower-the-views-of-education-professors-circa-2 0 10/) 
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How adequately does program coursework address "Analytical Skills”? 


35 



Virtually 

Very 

Limited 

Partially 

Adequate 

none 

limited 

Coverage 

adequate 



Undergraduate elementary (n=72) ■ Graduate elementary (n=14) 

■ Undergraduate secondary (n=69) ■ Graduate secondary (n=24) 


Less than 1 percent of the programs in the sample cover Analytical Skills adequately, with an additional 8 percent doing so with 
partial adequacy. The vast majority of programs ( 92 percent) have no, very limited or limited coverage. 


The ratings are considerably poorer in this second domain. Over one-quarter of all of the programs (29 percent) and almost 
half of the graduate programs (45 percent) were completely deficient, and only one program was found adequate. 

The program deemed “adequate” provided: 

1. Instruction on analysis of data from both classroom and standardized assessments, and 

2. Practice that includes both field-based practice in teams and presentation of results in both quantitative and 
graphic displays. 

In general, coursework appears to provide only the most basic tools for analysis of assessment data and then primarily from 
classroom assessment. Only 71 programs (39 percent) address standardized testing in the context of data analysis, and 
fewer — 22 programs (12 percent) — have classwork or homework practice that exposes teacher candidates to the process 
of analyzing data from standardized assessment. In 72 programs (40 percent), teacher candidates have to demonstrate 
analytical proficiency through independent “capstone projects” in which they analyze classroom assessment data, but this 
exercise appears to be candidates’ only practice in 43 programs (60 percent of those programs with such projects). We 
found evidence of collaborative analysis in in-class or capstone project assignments in a very small fraction (4 percent) of 
the programs. 
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How adequately does teacher preparation program coursework address 
Domain #3, Instructional Decision Making ? 

Instructional Decision Making 

An understanding of how to derive instructional guidance from assessment data. For example, a teacher candidate preparing 
to teach secondary mathematics might examine a variety of test results obtained in field work to determine the reasons 
students have performed poorly on a math problem on the subject of rates. The teacher candidate will need to determine 
if students lacked a basic understanding of rates themselves or if they were confused about the mixed-number multiplication 
necessary to compute the rate.* The next steps would depend on the outcome of this examination. They might lead to 
formative assessment to “zero in" on students' understanding either of rates or mixed-number multiplication (or both), 
appropriately focused instruction, and a summative assessment for final student evaluation. 

Appendix 2 contains a more detailed description of this domain, the rubric used for evaluating programs in this domain 
area and descriptions of the relevant features of sample programs in each of the rubric's rating categories. 

* See Driven by Data: A Practical Guide to Improve Instruction by Paul Bambrick-Santoyo (John Wiiey & Sons, Inc., 2010) for more examples of 
the triangulation approach to Instructional Decision Making. 

How adequately does coursework address "Instructional Decision Making"? 


35 



none limited adequate 

Coverage 

Undergraduate elementary (n=72) ■ Graduate elementary (n= 1 4) 

■ Undergraduate secondary (n=67) ■ Graduate secondary (n=24) 

Fewer than 2 percent of programs in the sample cover Instructional Decision Making adequately, with 7 percent doing so with partial 
adequacy. The vast majority of programs (91 percent) have no, very limited or limited coverage. 

In this third domain of Instructional Decision Making, findings remain almost as troubling as in the second domain. 
Only three programs (all undergraduate secondary) earned the highest score. In only a slight improvement over results 
in the second domain, almost one-quarter (23 percent) of all programs and over one-third (39 percent) of graduate 
programs were completely deficient. 
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Programs found adequate provided instruction and practice on using assessment data to drive 
instruction that is extensive and includes coursework that is subject-specific. 37 

Many programs do provide at least cursory exposure to the concept of “formative assessment,” 
which constitutes a toehold on how to use assessment data for planning. 38 Programs generally, 
however, do not appear to use their methods courses as the vehicles for more nuanced, subject- 
specific practice. In 42 elementary programs (48 percent of such programs) , using assessment 
to drive instruction appeared to be virtually absent from methods courses, meaning that no 
expert taught teachers how to examine assessment data with the benefit of subject matter- 
specific insight. 39 For example, a science methods course and a mathematics methods course 
could provide different but equally valuable insights into subtle misconceptions and 
misinterpretations that might underlie incorrect answers, or how assessment results might 
suggest necessary instructional scaffolding. 


In 54 secondary programs (58 percent of such programs), there was either no subject-specific 
methods course, or there was a subject-specific methods course that did not address assessment. 
Again, this finding means that a teacher candidate in a social studies certification route, for 
example, may not have the advantage of an instructor with content expertise as she tries to 
evaluate the content validity of a test she has developed with original multiple choice questions. 
Even in programs in which assessment was a topic of methods coursework, it was generally a 
very minor topic at best. For example, only one lecture might address assessment in the subject 
at hand; teacher candidates might be required to complete an assignment entailing only one 
assessment, with no explicit connection to how its use might inform instruction. 


The bar to earn 
a passing rating 
in this study was 
set low. 


Lastly, we found evidence of collaborative Instructional Decision Making practice in in-class 
or capstone project assignments in a very small fraction (8 percent) of the programs. 


How adequately does teacher preparation coursework address all 
three domains of assessment? 

Before discussing the overall performance of programs, it is important to note that the bar to 
earn a passing rating in this study was set low. 

Just how low? One relevant objective or lecture was sufficient to categorize a program as 
addressing standardized assessment. Several assignments were sufficient to categorize a program 
as having “adequate practice” in a particular area. Some might reasonably suggest that teachers, 
especially in this era of accountability and assessment, need far more exposure and practice 
than this standard suggests — a fair criticism in our judgment. 

37 This standard pertains to secondary programs. The analogous standard for elementary programs 
requires that instruction and practice are evident in all four core elementary subjects: language 
arts, mathematics, science and social studies. 

3 8 See Appendix 5 for a tutorial on assessment taxonomy. 

39 As mentioned earlier, we assume that virtually all methods courses address assessment to the 
minimal extent of requiring that lesson plans include at least summative assessments. 
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Why do we set such a low bar?We do so to give institutions the benefit of the doubt. Though 
our examination of course materials is comprehensive, our low bar compensates for our inability 
to audit the actual course and the risk we face that a course might actually cover material that 
was not spelled out in the syllabus. This approach means that some programs earn higher 
marks than perhaps they should have. But it also means that our margin of error is so substantial 
that there should be little doubt that a program designated as inadequate is in fact inadequate. 

Our overall conclusion is that while assessment is addressed to some extent in all but five of 
the 180 programs we examined, only six programs (3 percent) provide preparation that can 
be deemed adequate: four elementary programs and two secondary programs. 


Only six programs 
(3 percent) provide 
preparation that 
can be deemed 
adequate. 


When it comes to assessment, what is an "adequate" program? 

1) The program provides a foundation in Assessment Literacy, with attention to the 
instructional role of standardized tests, particularly the program state's standardized tests; 

2) The program requires teacher candidates to prepare formative and summative classroom 
assessments; and 

3) The program builds a relatively strong foundation for interpreting and applying data, both 
individually and in teams, from standardized and classroom assessments in order to inform 
and adjust instruction. 

These three parts are based roughly on the three domains, but combining them into an overall 
metric moves us away from a simple aggregation of ratings in each domain. Frankly, this allows 
us to note a program as “adequate" that may not have received the highest rating on any of 
the domains but still provides a “relatively strong” foundation in all. 


How adequate overall is teacher preparation program coverage of assessment? 



Only 3 percent of programs in the sample cover assessment adequately, with 7 3 percent doing so 
with partial adequacy. Over three-quarters of programs (83 percent) have inadequate coverage. 


18 



FINDINGS 


The most common weaknesses: 


■ As noted, programs were rated much more poorly on the second (Analytical Skills) and 
third (Instructional Decision Making) domains than on the first (Assessment Literacy) . 
Even among the programs earning the highest rating on the first domain, few were 
adequate on the domains of Analytical Skills and Instructional Decision Making. 

■ While about two-thirds of programs address standardized assessments, coverage is 
found largely in the first domain of Assessment Literacy Even programs that address 
standardized assessments rarely discuss data from standardized tests in the context of 
Analytical Skills or Instructional Decision Making, or have relevant practice. 

■ Programs rarely mention the use of data by teachers for school improvement purposes, 
(as opposed to improvement of the performance of their own students) . 

■ There is little evidence of collaborative practice being modelled or promoted. 

We found that programs in the same institution may be internally inconsistent: 

In only one of the three institutions in which we evaluated all four program types were all 
the programs rated the same — in this case, unfortunately, as “inadequate.” 

The impact of state regulation. 

In many areas of teacher preparation, NCTQ has found that state regulations governing the 
content of teacher preparation have little impact in actual practice. 40 In the policy brief we 
released earlier this year, our analysis of a smaller sample of programs suggested that state 
regulation on preparation for assessment might be an exception to this general rule. With the 
full sample in hand, however, we can once again confirm that state regulations do not affect 
what teachers learn in their preparation programs. 41 As noted in our recommendations, this 
is likely due to the fact that states do not require candidates to pass tests in this subject to 
become licensed. 


Even among the 
programs earning 
the highest rating 
on the first domain, 
few rated well on 
the domains of 
Analytical Skills 
and Instructional 
Decision Making. 


40 See Appendix 4 for a discussion of relevant state regulations. 

41 The 1 80 programs evaluated to date are found in 30 states, 1 8 of which have regulations that at least 
address to some extent the responsibility of teacher preparation programs to address assessment. 
(See graphic in Appendix 4.) In all states, whether regulations are relatively strong or weak, 
ratings on programs within the state are mixed in states for which more than one program was 
evaluated. However, only a slightly higher proportion of programs are “adequate” or “partially 
adequate” ( 1 8 percent) in states that are relatively stronger on regulation than in states that are 
not ( 1 5 percent) . 
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The impact of accreditation. 

In previous analyses of various areas of teacher preparation we have not found a significant difference between accredited 
and non-accredited institutions. In this area, we have, and the difference points to a negative impact of accreditation 
for the 157 programs that are accredited by either NCATE orTEAC out of this sample of 180 programs. 42 


Does accreditation make a difference? 



Accredited Not accredited 

Programs rated adequate 
or partially adequate 


Twenty-six percent of the programs in non-accredited 
institutions are rated as "adequate" or "partially adequate, 
whereas only 75 percent of the programs in institutions 
accredited by NCATE or TEAC fall into either of these two 
rating categories. 


Does a state's adoption of InTASC professional 
standards make a difference? 



Programs Programs in 

in InTASC states that are 

member states not InTASC 
members 

Programs rated adequate 
or partially adequate 

Only 9 percent of the programs in states that are members of 
InTASC are rated "adequate" or "partially adequate. "A much 
higher proportion (30 percent) of programs in states that are 
not members fall into either of these two rating categories. 


The impact of professional standards 

As with accreditation, if there is a difference, it appears to indicate a negative impact of professional standards for the 
109 programs in the 2 1 states in the sample which are members of InTASC and subscribe to its professional standards. 43 


42 See Appendix 4 for a discussion of relevant accreditation standards. 

43 See Appendix 4 for a discussion of relevant professional standards. 
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Recommendations 

We fully recognize that assessment and use of assessment data is such a large and important topic for teacher candidates 
that no matter how adequate a program may be in laying a foundation, continued training will be necessary to build 
and hone knowledge and skills during induction and throughout classroom teaching. But the need for in-service training 
does not negate the need for foundational knowledge to be also laid during pre-service. Numerous levers are available to 
provide both increased pressure on and increased support for teacher preparation programs to lay a better foundation for 
continuing professional development in the area of assessment. 

Federal Government 

Provide more legislative guidance 

The federal government should encourage better teacher candidate preparation in data driven instruction through: 

1 . Amending Title II of the Higher Education Act (HEA) , providing suitable incentives for teacher preparation programs 
to prepare teacher candidates to understand and use assessments and assessment data; 

2 . Requiring that a portion of State Longitudinal Data System (SLDS) funds be used for training on assessment and 
data skills; and 

3. Allocating a percentage of funds under Title II of the Elementary and Secondary Education Act (ESEA) for training. 
Amendments to HEA and ESEA, respectively, could also 

1. Provide states with incentives to ensure that their program approval authority reinforces the importance of 
teacher data skills; and 

2. Encourage innovative state and local practices in data driven instruction. 

Invest in research 

There are a number of issues related to data driven instruction that warrant exploration. Federal policy should support 
research into its effects on student achievement, the knowledge and skills teachers need to become expert, the conditions 
within schools necessary for it to have maximum impact, the collection and dissemination of best practices related to 
pre-service and in-service training, and the compilation and meta-analysis of existing research on assessment, data driven 
instruction and related topics. 
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Now — of necessity 
— districts must 
hire new teachers 
who no sooner 
arrive at their new 
schools than they 
need professional 
development on a 
variety of assessment 
tools and tasks 
necessary for school 
improvement. 


We present two different and potentially complementary approaches that might increase the 
accountability of preparation programs, one for states and one for school districts: 

States 

Increase accountability of preparation programs 

Teacher licensing tests are perhaps the most effective and efficient means available to drive 
significant change in teacher preparation programs. One possible reason that assessment is 
addressed so poorly in teacher preparation programs is that it is not covered at any depth 
in the licensing tests required by most states. As a first step to remedying this situation, the 
Council of Chief State School Officers (CCSSO) could evaluate the scope of topics addressed 
in those licensing tests that do include assessment to make recommendations to states about 
how licensing tests might be improved in this area. If warranted, CCSSO could sponsor a 
consortium of states who wish to aggressively promote licensing test enhancements. 

School Districts 

Ensure a better applicant pool for hiring 

Of all the stakeholders in the country’s educational system, school districts have the greatest 
vested interest in hiring teachers with a solid foundation in assessment. Now — of necessity 
— districts must hire new teachers who no sooner arrive at their new schools than they need 
immediate professional development on a variety of assessment tools and tasks necessary 
for school improvement. Districts hiring a large share of graduates from any given teacher 
preparation program might use their bargaining power to bring the program to the table to 
ensure that its teacher candidates arrive with a better foundation in all assessment domains. 

Absent a state effort to focus pedagogical licensing tests on the topic of assessment knowledge 
that teachers need, we recommend that districts administer their own test, one that applicants 
must take as a condition of hiring. Such a test might be designed under the supervision of, 
for example, the Council of the Great City Schools. It may be the most efficient step toward 
persuading states that assessment knowledge needs to be an integral part of any licensing test. 

Foundations 

Develop instructional data sets and model curricula 

A variety of graduate programs for teachers and administrators offer coursework in the data 
driven instruction model that can be taken in degree programs or as stand-alone in-service 
training . 44 Participants must bring to these programs their own district’s student performance 
data inventory. Course assignments have participants working directly with their own data. 
Teacher educators in initial certification programs seeking to improve their courses, however, 


44 As mentioned earlier, all teachers in the Charlotte-Mecklenburg school district — a Broad Prize winner 
- have taken one of the most well-known courses, Harvard’s “Data Wise” course. 
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would have a hard time replicating these inventories. Neither the instructor nor enrolled teacher 
candidates has access to the appropriate data, and fabricating data is not easy. 45 

Innovation-minded teacher educators would welcome synthetic but realistic elementary 
and secondary school data sets that contain student performance data from a typical range 
of classroom, district and state assessments. These data sets could form the basis of a very 
rich collection of simulations and exercises designed to develop expertise in Assessment Literacy, 
Analytical Skills and Instructional Decision Making in existing coursework or in model course 
curricula developed by foundations and their partners. 46 

How might student performance data be used in instruction? 

Consider as a hypothetical a case example in an Institute for Education Sciences practice guide 
on “Using State Achievement Data to Support Instructional Decision Making."* The case example 
describes how a group of elementary teachers might analyze data on student performance in 
mathematics: The teachers find that students are weakest on state tests in geometry skills 
related to shapes and measurements, but data on district interim assessments do not reveal 
the source of the problem. It is only when student workbooks are consulted that the teachers 
surmise that the students' difficulties relate to the need to combine shapes for some calculations. 
This leads the teachers to generate some relevant classroom lessons. The next interim district 
test shows improvement on the type of problems that the teachers had analyzed collaboratively. 

Teams of teacher candidates could simulate this same exercise in coursework. They could be 
provided a data set containing 1) fabricated but realistic 4th grade student test scores on the 
mathematics section of a state test and interim district assessments taken throughout the 
year, 2) appropriate statistical descriptions and graphic displays, and 3) examples of student 
work designed to illuminate possible student failings and misconceptions. Working together, 
the teacher candidates could develop appropriate lesson and assessment plans, just as they 
might soon be doing with colleagues when they begin teaching. 

Developing data sets for the core subjects at both the elementary and secondary levels could 
provide ample practice for teacher preparation coursework to be used ideally in courses in 
which faculty with assessment expertise and faculty with pedagogical expertise could coordinate 
instruction or team-teach. 

* National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences. (September 
2009). IES practice guide: Using student achievement data to support Instructional Decision Making (p. 1 7). 
Washington, DC: U.S. Department of Education. 


45 Several instructors of graduate level assessment coursework informed us that they have attempted 
to develop such a data inventory for course participants who had no access to data and that they 
had not found the results satisfactory because the task of replicating the massive amount of data 
now available to many school administrators and teachers is so difficult. 

46 The DQC reports that discussions are underway between the New Hampshire Department of 
Education and teacher preparation programs in the state to develop a college course around 
effective use of data to plan instruction. (Access at: http:/ /www.dataqualitycampaign.org/ 
resources/ field_profiles/NH.A9) 


School data sets 
that contain student 
performance data 
from a typical 
range of classroom, 
district and state 
assessments could 
form the basis 
of simulations 
and exercises in 
teacher preparation 
coursework. 
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Conclusion 

We find evidence of a serious misalignment between what school districts seek in the skills of their teacher corps and 
the training provided by teacher preparation programs. It is noteworthy that only two of the 45 5 assessment courses 
we identified appear to exploit local school district expertise, in this case by having a representative from a local district 
lecture, presumably on student performance data in the district context. 

In too many programs, assessment coursework centers only on the tests that teachers have always administered, preparing 
teacher candidates to develop and use assessment data to improve their students’ performance in an insular environment. 
As important as this type of preparation continues to be, it shortchanges teacher candidates because it does not 
represent the environment of schools in this century. New teachers undoubtedly find themselves confronting 
data presentations using unfamiliar terms and concepts. The flow of data they currently have to deal with will only 
grow larger in 2014, as new assessments tied to the Common Core state standards are first administered. Today’s 
schools demand teachers who can comfortably understand and utilize — both individually and collaboratively — a 
full range of classroom and standardized data, whether the data relate to their own students or to all the students in 
their school. Preparing them for anything less is unfair to teacher candidates, as well as to the many students they plan 
to teach. 
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Appendix 1: Institutions included in the sample 

Undergraduate Undergraduate Graduate Graduate 


Institution 

State 

Elementary 

Secondary 

Elementary Secondary 

Adams State College 

CO 

X 

X 

Alcorn State University 

MS 

X 

X 

Alice Lloyd College 

KY 


X 

Appalachian State University 

NC 

X 

X 

Ball State University 

IN 



Binghamton University, State University of New York 

NY 


X X 

Bloomsburg University of Pennsylvania 

PA 


X 

Blue Mountain College 

MS 

X 

X 

Boise State University 

ID 

X 

X 

Catawba College 

NC 

X 

X 

Central Michigan University 

Ml 

X 

X 

Central Washington University 

WA 

X 

X 

Chadron State College 

NE 

X 

X 

Chipola College 

FL 

X 

X 

Clayton State University 

GA 


X 

Colorado Mesa University 

CO 

X 

X 

Colorado State University-Pueblo 

CO 

X 

X 

Dalton State College 

GA 

X 


Daytona State College 

FL 

X 

X 

Dickinson State University 

ND 


X 

East Carolina University 

NC 

X 

X 

Eastern Kentucky University 

KY 

X 

X 

Elizabeth City State University 

NC 


X 

Elon University 

NC 

X 

X 

Emporia State University 

KS 

X 

X 

X 

Florida A&M University 

FL 

X 

X 

Florida State University 

FL 

X 

X 

X X 

Fort Hays University 

KS 


X 

Great Basin College 

NV 

X 

X 

Harris Stowe State University 

MO 

X 


High Point University 

NC 

X 

X 

Hunter College - CUNY 

NY 


X X 

Indian River State College 

FL 


X 

Indiana State University 

IN 

X 

X 

Indiana University - Purdue University Indianapolis 

IN 

X 

X 

Indiana University Bloomington 

IN 

X 

X 

Indiana University East 

IN 

X 

X 

Indiana University Kokomo 

IN 

X 

X 

Indiana University Northwest 

IN 

X 

X 

Indiana University South Bend 

IN 

X 

X 
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Undergraduate Undergraduate Graduate Graduate 


Institution 

State 

Elementary 

Secondary 

Elementary 

Secondary 

Indiana University Southeast 

IN 

X 

X 



Kentucky State University 

KY 

X 

X 



Lake Superior State University 

Ml 

X 




Lehman College - CUNY 

NY 



X 

X 

Lewis-Clark State College 

ID 

X 

X 



Mayville State University 

ND 

X 

X 



Miami-Dade College 

FL 


X 



Michigan State University 

Ml 


X 



Michigan Tech University 

Ml 


X 



Minot State University 

ND 

X 

X 



Mississippi University forWomen 

MS 

X 



X 

Morehead State University 

KY 

X 

X 



Murray State University 

KY 

X 

X 



North Carolina A&T State University 

NC 

X 




North Carolina State University 

NC 

X 


X 


Northeastern State University 

OK 

X 

X 



Northern Kentucky University 

KY 

X 

X 


X 

Northern State University 

SD 

X 




Northwest Florida State University 

FL 

X 




Northwestern Oklahoma State University 

OK 

X 

X 



Oklahoma Baptist University 

OK 

X 

X 



Penn State 

PA 

X 



X 

Penn State Harrisburg 

PA 

X 




Peru State College 

NE 

X 

X 



Pittsburg State University 

KS 

X 

X 


X 

Plymouth State University 

NH 

X 



X 

Potsdam, The State University of New York 

NY 

X 

X 

X 

X 

Rutgers University - Camden 

Nj 

X 

X 



Saginaw Valley State University 

Ml 

X 

X 



St. Mary's College of Maryland 

MD 



X 

X 

St. Cloud State University 

MN 

X 

X 


X 

St. Petersburg College 

FL 



X 


SUNY Oneonta 

NY 


X 



The University of Southern Mississippi 

MS 

X 

X 



The University of Tennessee Knoxville 

TN 



X 

X 

The University of Utah 

UT 

X 




University of Alaska Fairbanks 

AK 

X 




University of Colorado Denver 

CO 

X 

X 

X 

X 

University of Central Oklahoma 

OK 


X 



University of Cincinnati 

OH 

X 




University of Colorado at Colorado Springs 

CO 

X 

X 



University of Colorado Boulder 

CO 

X 

X 
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Undergraduate Undergraduate Graduate Graduate 


Institution 

State 

Elementary 

Secondary 

Elementary 

Secondary 

University of Kentucky 

KY 

X 

X 


X 

University of Louisville 

KY 

X 

X 

X 


University of New Hampshire 

NH 



X 

X 

University of North Carolina - Charlotte 

NC 



X 

X 

University of South Dakota 

SD 

X 

University of Virginia 

VA 



X 

X 

Vicennes University 

IN 

X 

Wayne State College 

NE 

X 

X 



West Virginia University at Parkersburg 

WV 

X 

Western Carolina University 

NC 

X 

Western Kentucky University 

KY 

X 

X 


X 

Western Michigan University 

Ml 

X 

X 

X 


Western Washington University 

WA 

X 

Winthrop University 

SC 

X 

York University - CUNY 

NY 

X 

X 

X 


Youngstown State University 

OH 


X 

X 


Totals 


73 

69 

14 

24 
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Appendix 2: Evaluating for the three assessment domains 

For each program, we rated coursework on the degree to which it delivered instruction in the three domains of 
Assessment Literacy, Analytical Skills and Instructional Decision Making. Using the rubrics below, we assigned a rating 
between “0” and “4” to each program. 47 

Note that in the first two rubrics, a determination that a program delivered instruction with a “comprehensive scope” 
means that instruction covered both classroom assessments and standardized assessments, including district, state and 
national tests. A program with at least two practice assignments was deemed to have “adequate practice.” 

Evaluating for Assessment Literacy 

More about Assessment Literacy 

Understanding the many different categories and forms of assessment is fundamental to the capacity to generate or 
select appropriate classroom assessments and to understand the rationales for the many different types of district, state 
and national assessments. The teacher candidate should 1) understand the taxonomy of assessment; 2) review a variety 
of classroom assessments, and district and state standardized tests; and 3) practice developing a variety of classroom 
assessments. Teacher candidates should also gain an understanding of assessment “bias” and what “validity” and “reliability” 
mean with respect to assessment items. 

The taxonomy of assessment in and of itself is a challenging topic, but is also absolutely essential. A typical course might 
organize assessments in a simple list (e.g, norm-referenced, criterion-referenced, formal, informal, formative, summative, 
diagnostic, performance, selected response, open response and portfolio), but teacher candidates need to be able to 
organize these assessments along dimensions such as purpose (formative vs. summative), approach (authentic vs. 
traditional), and scoring (norm-referenced vs. criterion-referenced) . They also need to understand how validity and 
reliability issues connect to these categorizations. Without this grounding, a teacher candidate cannot understand why 
one state assessment might assert that results cannot be used to determine instructional objectives for a 3rd grade 
class, while a district assessment’s results might be intended for that very purpose. An assessment taxonomy tutorial 
found in Appendix 5 illustrates the assessment topics that teacher preparation coursework should address and may also 
inform the reader about types of assessment. 

Rubric for evaluating Assessment Literacy (elementary and secondary programs) 

Instructional material considered for rating on this rubric: course objectives, textbooks and required readings, lectures, 
practice, and practice aligned with field work, including capstone projects. 


47 A rating of “Could not be determined” (CBD) was given if the nature of instruction and/ or practice could not be ascertained 
using syllabus material. The program was removed from the sample if CBD ratings in one or more domains precluded devel- 
opment of an overall rating. 
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0 

1 

2 

3 

4 

There is no or almost no 

Instruction on the various 

Case 1: The scope of 

The scope of instruction 

The scope of instruction 

instruction or practice 

types of assessment is 

Instruction on the various 

on various types 

on the various types 

on the various types of 

very limited and there is 

types of assessment is 

of assessment is 

of assessment is 

assessment. 

no or almost no practice. 

not comprehensive and 
practice is very limited to 
adequate. 

OR 

Case 2: The scope of 
instruction on the vari- 
ous types of assessment 
is comprehensive, but 
practice is very limited or 
limited. 

comprehensive and 
there is adequate 
practice. 

comprehensive, 
including concepts 
such as “validity" and 
“reliability,” and there 
is adequate practice. 


Examples of evaluations in Assessment Literacy at each rating level 

Note: Each example is provided for illustrative purposes only and represents only one of many ways by which coursework 
could earn a particular rating. 

Sample program earning a “0”: There is no or almost no instruction or practice on the various types of assessment, 
m The focus is on classroom assessment only. 

■ Only one assessment-relevant course is in the program; no objectives in this course’s syllabus mention assessment. 

■ “Assessment in the Classroom” is the topic of the one lecture on assessment. 

■ The text on educational psychology is presumed to cover assessment, but only very generally. 

■ There are no practice assignments related to assessment and no required capstone project. 

Sample program earning a “1”: Instruction on the various types of assessment is very limited, and there is no or 
almost no practice. 

m The focus is on classroom assessment only. 

■ Only one assessment-relevant course is in the program; of six objectives, only one explicitly pertains to assessment: 
Candidates will be able to develop, implement, and utilize curriculum that encompass a variety of assessment methods. 

u Assessment is the topic of one lecture. 

■ There are no required textbooks or readings that address any aspect of assessment. 

■ One practice assignment relates to assessment: Teacher candidates develop a curriculum unit with a pre- and 
post-test as a component of their held work. 

Sample program earning a “2” (case 1) : The scope of instruction on the various types of assessment is not comprehensive 
and practice is very Hmited to adequate. 

■ The focus is on classroom assessment only. 

■ Two courses are assessment-relevant, each with one explicit measurement-related objective. 

■ There is one lecture entitled “How do I know what my students know?” 

■ There are no required textbooks or readings that address any aspect of assessment. 

■ Practice assignments involving developing assessments take place in field work associated with coursework, 
student teaching and a capstone project. 
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Sample program earning a “2” (case 2): The scope of instruction on the various types of assessment is comprehensive, 
but practice is very limited or limited. 

m Both classroom and standardized assessment are addressed. 

■ Three courses are assessment-relevant, and two of these have five measurement-related objectives. 

■ Among the three courses, there are seven lectures on assessment topics, including “Large Scale Standardized 
Tests and the Classroom” and “Formative Assessment: Ongoing Assessment to Promote Student Success.” 

■ A required textbook addresses assessment. 

■ Practice assignments involve evaluation of a standardized test and an “integrated instructional plan,” including 
a strategy for assessing state/ district standards, but there are no assessment assignments associated with field 
work (considering both student teaching and work associated with a capstone project). 

Sample program earning a “3”: The scope of Instruction on the various types of assessment is comprehensive, 
and there is adequate practice. 

m Both classroom and standardized assessment are addressed. 

■ Five courses are assessment-relevant, and three of these have six explicit measurement-related objectives. 

■ Among the five courses, there are five lectures on assessment topics, including “Standardized Testing and 
Tests” and “Formative/Informative and Summative Assessment.” 

■ Two required textbooks address classroom assessment, with other required readings possibly addressing assessment 
more broadly. 

■ Coursework includes practice on curriculum-based assessment, and a capstone project completed in student 
teaching requires teacher candidates to create a plan for evaluation of student learning that includes pre-assessment, 
formative assessments and summative assessments. 

Sample program earning a “4” : The scope of instruction on the various types of assessment is comprehensive, including 
concepts such as "validity” and “reliability, and there is adequate practice. 

m Both classroom and standardized assessment are addressed. 

■ While only one course is assessment-relevant, six of its objectives relate to measurement. 

■ The syllabus indicates that the course will cover “local, state and national assessment requirements,” a scope 
of coverage that is underscored by the objectives that address a comprehensive range of types of assessments 
(including NAEP) , and include mention of reliability and validity. 

■ Two required textbooks address assessment. 

■ A “work sample” of assessments includes an example of each of six types of formats, and an assessment 
project conducted during student teaching includes development of a variety of assessments. 
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Evaluating for analytic skills 

More about Analytical Skills 

The teacher candidate should understand how to dissect, describe and display the data that emerge from assessments. 48 
For classroom assessment, the tools may be fairly simple. The descriptive statistics may involve no more than a discussion 
of means, medians or modes, and the graphic displays can be simple. A teacher candidate might develop a line graph 
that demonstrates whether an achievement gap between the lowest and highest achieving groups of students in a class 
has been reduced over the course of a cycle of formative assessment, instruction, re-teaching and remediation, and 
final summative assessment. 

To understand district and state assessments, the teacher candidate should understand the concepts of sampling error 
and measurement error, different types of scores (raw scores, percentiles, cut-scores, performance levels, grade equivalent 
scores, developmental scale scores), how scores are used to determine student growth or student “value-added” 
growth, and how data from various types of assessments can be used in conjunction to triangulate on causes for poor 
student performance. 


Rubric for evaluating Analytical Skills (elementary and secondary programs) 

Instructional material considered for rating on this rubric: course objectives, textbooks and required readings, lectures, 
practice, and practice aligned with field work (including capstone projects), with consideration of the extent of 
collaborative practice. 


0 

There is no or almost 
no practice or instruction 
preparing teachers 
to analyze data from 
assessments. 


1 

Instruction for preparing 
teachers to analyze data 
from assessments is 
very limited, and there 
is almost no or very 
limited practice. 


2 

Case 1:The scope of 
instruction on analyzing 
data from assessments 
is not comprehensive, 
there is limited to 
adequate practice that 
includes at least one of 
the following features: 
field-based practice, 
presentation of results 
in quantitative and 
graphic displays and/or 
analysis and presentation 
in teams. 

OR 

Case 2: The scope of 
instruction on analyzing 
data from assessments 
is comprehensive, but 
practice is very limited 
or limited. 


3 

The scope of instruction 
to prepare teachers 
to analyze data 
from assessments is 
comprehensive, and 
there is adequate practice 
that includes at least one 
of the following features: 
field-based practice, 
presentation of results 
in quantitative and 
graphic displays and/or 
analysis and presentation 
in teams. 


4 

The scope of Instruction 
on analyzing data 
from assessments is 
comprehensive, and 
there is adequate practice 
that includes both of 
the following features: 

1) field-based practice 
in teams and 2) 
presentation of results 
in quantitative and 
graphic displays. 


48 As mentioned earlier, when teachers actually work with data at the school level, student data include both performance data 
and student record data (such as attendance information) that are relevant to performance. 
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Examples of evaluations in Analytical Skills at each rating level 

Note: Each example is provided for illustrative purposes only and represents only one of many ways by which coursework 
could earn a particular rating. 

Sample program earning a “0” : There is no or almost no practice or instruction preparing teachers to analyze data 
from assessments. 

m The focus is on classroom assessment only. 

■ While there are two assessment-relevant courses, there is only one objective on the topic of analysis of assessment data. 

■ There are no lectures on analysis of assessment data. 

■ There are no required textbooks or readings addressing assessment or assessment data. 

■ The only practice is practice aligned with field work, which requires only that the teacher candidate teach a 
lesson and “assess change in understanding and submit a report detailing your experience and reflections.” 
There is no collaborative practice. 

Sample program earning a “ 1 ” : Instruction on analyzing data from assessments is very limited, and there is almost 
no or very limited practice. 

m The focus is on classroom assessment only. 

■ There are three assessment-relevant courses that together have three objectives on the topic of analysis of 
assessment data. 

■ There is one lecture on “assessment analysis.” 

■ There are no required textbooks, but there is one required reading on “making sense of assessment.” 

■ The only practice is practice aligned with field work, which requires candidates to analyze an assessment. 
There is no collaborative practice. 

Sample program earning a “2” (case 1): The scope of instruction on analyzing data from assessments is not 
comprehensive, but there is limited to adequate practice that includes at least one feature: field-based practice, 
presentation of results in quantitative and graphic displays and/or analysis and presentation in teams. 

m The focus is on classroom assessment only. 

■ There are two assessment-relevant courses that together have only one objective on the topic of analysis of 
assessment data. 

■ Course topics include “communicating assessment result.” 

■ There is a required textbook that addresses assessment types comprehensively, but it does not appear to address 
analysis of assessment data. 

■ Practice involves both individual analysis (“After teaching your lesson in the field setting, you will submit a 
reflective paper that analyzes the effectiveness of the lesson in terms of student learning”) and collaborative 
group analysis (“You will be presented with the results of student performance on an exam and be required 
to analyze the data. . .This will be a group project completed in class”). 


National Council on Teacher Quality | 35 




WHATTEACHER PREPARATION PROGRAMS TEACH ABOUT K-12 ASSESSMENT 


Sample program earning a “2” (case 2) : The scope of instruction on analyzing data from assessments is comprehensive, 
but practice is very limited or Hmited. 

m Both classroom and standardized assessment are addressed. 

■ There is one assessment- relevant course that has one objective on the topic of analysis of assessment data. 

■ The course addresses “summarizing and communicating assessment data.” 

■ There is a required textbook, but it does not appear to focus on analysis of assessment data. 

■ The only practice is two assignments during field work: 1) an assignment to “document a selected student’s 
performance using existing classroom assessments, school-based state assessment data (CATS), school-based 
national standardized assessment (CTBS) and other assessment sources and to communicate the student’s 
progress in a form that is clear to a parent, team teacher, and counselor or principal;” and 2) an analysis of 
assessment data completed during student teaching. There is no collaborative practice. 

Sample program earning a “3”: The scope of instruction to prepare teachers to analyze data from assessments 
is comprehensive, and there is adequate practice that includes at least one of the following features: field-based 
practice, presentation of results in quantitative and graphic displays, and/or analysis and presentation in teams. 

m Both classroom and standardized assessment are addressed. 

■ There is one assessment-relevant course that has two objectives on the topic of analysis of assessment data, 
one of which is unusually technical: “Interpret standardized test scores (i.e., percentile, raw score, scaled 
score, grade equivalent score, stanine) .” 

■ One lecture focuses on interpreting assessment results. 

■ The required textbook addresses assessment types comprehensively, but it does not appear to focus on data 
analysis. 

■ Practice includes two assignments aligned with field work: 1) “Using the summative assessment data from 
the unit you taught, you will present the data as a whole-class graph, and also as a ’gap-group’ graph, accompanied 
by descriptive statistics and a paragraph explaining the data and possible interpretations regarding student learning”; 
and 2) As part of a comprehensive unit plan, the instructor will grade “how you will communicate your 
students’ progress to the students and their parents.” There is no collaborative practice. 

Sample program earning a “4”: The scope of Instruction on analyzing data from assessments is comprehensive, 
and there is adequate practice that includes both of the following features: 1) field-based practice in teams and 2) 
presentation of results in quantitative and graphic displays. 

m Both classroom and standardized assessment are addressed. 

■ There is one assessment-relevant course that has three objectives on the topic of analysis of assessment data, 
one of which indicates that the teacher “should be skilled in administering, scoring and interpreting results 
of both externally-produced and teacher-produced assessment methods.” 

■ One lecture focuses on analysis of student learning. 

■ Two required textbooks focus on assessment data and communicating assessment results. 

■ There is collaborative practice to “prep” for separate individual analysis of student learning and a strong 
assignment on analysis of data in a capstone project that must be accompanied by a discussion of how colleagues 
provided aid in analyzing and interpreting individual student work products and group growth. 
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Evaluating for Instructional Decision Making 

More about Instructional Decision Making 

The teacher candidate needs to begin to practice how to derive instructional guidance from assessment data. Because 
the purpose of formative assessment is to modify teaching and learning activities as necessary to improve student 
performance, practice using formative assessment is by definition practice in this area. 49 Programs teaching processes 
such as “Understanding by Design” or “Curriculum-Based Evaluation” are attempting to develop this “backwards” 
thinking approach to instruction. That approach encourages teachers to develop instruction based on clear indications 
of what students already know and what they still need to master. 

While it is possible to deal with the fundamentals of assessment-informed planning in the abstract through exposure 
to and practice with formative assessments, more sophisticated instruction should be provided by subject-matter experts 
who teach methods courses. Because there can be a variety of reasons why a student gives an incorrect answer to an 
assessment question, learning how to discern the instructional implications from assessment data goes well beyond using 
a formative assessment. Purposeful and directed examination of a variety of classroom assessment data and standardized 
data is sometimes necessary to identify the source of misunderstandings by students. 

Rubric for evaluating Instructional Decision Making (elementary programs only) 

Instructional material considered for rating on this rubric: The range of subjects addressed in methods coursework, 
course objectives, lectures, textbooks and required readings, practice, and practice aligned with field work (including 
capstone projects), with consideration of the extent of collaborative practice. 


0 

1 

2 

3 

4 

There is no or very limited 
instruction or practice 
that prepares teachers 
to use assessment data 
to drive instruction in 
specific elementary 
subject areas.* 

There is limited 
instruction or practice 
that prepares teachers 
to use assessment data 
to drive instruction in 
specific elementary 
subject areas.* 

Instruction and practice 
on using assessment data 
to drive instruction 
is evident but only in 
one or two elementary 
subject areas.* 

Instruction and practice 
on using assessment data 
to drive instruction is 
evident but not in ail four 
core elementary subject 
areas.* 

Instruction and practice 
on using assessment data 
to drive instruction is 
evident in all four core 
elementary subject 
areas.* 


* For purposes of this analysis, the four core elementary subject areas are the language arts, mathematics, science and social studies. 

Rubric for evaluating assessment-informed planning (secondary programs only) 

Instructional material considered for rating on this rubric: assessment coursework and secondary methods coursework 
in one selected pathway, course objectives, lectures, textbooks and required readings, practice and practice aligned 
with field work (including capstone projects), with consideration of the extent of collaborative practice. 


0 

1 

2 

3 

4 

There is no or almost no 
instruction or practice 
that prepares teachers 
to use assessment 
data to drive secondary 
instruction. 

There is very limited 
instruction or practice 
that prepares teachers 
to use assessment 
data to drive secondary 
instruction. 

Instruction and practice 
on using assessment 
data to drive instruction 
is limited but includes 
coursework that is 
subject-specific. 

Instruction and practice 
on using assessment 
data to drive instruction 
is evident and includes 
coursework that is 
subject-specific. 

Instruction and practice 
on using assessment 
data to drive instruction 
is extensive and includes 
coursework that is 
subject-specific. 


49 For purposes of our analysis, we counted instruction and practice on “pre-assessment” as instruction and practice on formative 
assessment, even though pre-assessment is only one of many forms of formative assessment. 
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Examples of evaluations in Instructional Decision Making at each rating level 

Note: Each example is provided for illustrative purposes only and represents only one of many ways by which coursework 
could earn a particular rating. 

Elementary programs 

Sample elementary program earning a “ 0 ” : There is no or almost no instruction or practice that prepares teachers 
to use assessment data to drive instruction in specific elementary subject areas. 

m Assessment is such a small feature of a course taken in conjunction with student teaching that there is only 
one general assessment-related objective and no objective related to assessment-informed planning. 

■ Lectures on the topic are limited, if provided at all. 

■ There is no required textbook addressing assessment in general or using assessment to inform planning. 

■ There is no practice (thus no practice aligned with field work or collaborative practice) on using assessment 
data to plan instruction. 

Sample elementary program earning a “1”: There is very limited instruction or practice that prepares teachers to 
use assessment data to drive instruction in specific elementary subject areas. 

m While a non-methods course in this program has four objectives on using assessment data for planning, it 
does not appear to have any related instruction on using assessment data to plan instruction. 

■ There is one lecture on “Formative Assessment: Ongoing Assessment to Promote Student Success.” 

■ Two methods courses are assessment-relevant, but neither has any coursework features that suggest instruction 
in this assessment domain. 

■ There is no practice (thus no practice aligned with field work) , nor is there collaborative practice on using 
assessment data to plan instruction. 

Sample elementary program earning a “2”: Instruction and practice on using assessment data to drive instruction 
is evident but only in one or two elementary subject areas. 

m The two assessment-relevant courses in this program include one focused on assessment only and a “curriculum 
and instruction” course that also touches on the topic; between the two courses, there are two objectives 
relevant to using assessment data to plan instruction. 

■ There are no lectures on using assessment data to plan instruction. 

■ A required textbook addresses the “use of assessment to support student learning.” 

■ There is one practice assignment, and it is aligned with field work: “How did the data collected from the 
pre-assessment guide lesson development and instruction? Based on your experience teaching the lesson and 
analyzing assessment data, what changes would you make in the lesson?” There is no collaborative practice. 
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Sample elementary program earning a “3”: Instruction and practice on using assessment data to drive instruction 
is evident but not in all four core elementary subject areas. 

m While a total of five courses in this program were deemed assessment relevant (including a social studies 
course, a math methods course and an elementary curriculum course), only one course has one objective 
related to using assessment data for planning instruction. 

■ One-half of one lecture is devoted to “Formative /Informative” assessment. 

■ Two required textbooks address assessment, and one of these focuses on “backwards design.” 

■ While there is no practice in this program other than in a capstone project, the practice required in the project 
is strong. It requires the teacher candidate to discuss “the ways in which formative and summative data 
informed instructional decisions. . . [with] implications for remediation and revision of instruction” and to “propose 
a remediation plan for those students who did not make growth.” 

Sample elementary program earning a “4”: No elementary program earned a “4” in this domain. 


Secondary programs 

Sample secondary program earning a “0”: There is no or almost no instruction or practice that prepares teachers 
to use assessment data to drive secondary instruction. 

m Neither of the two assessment-relevant courses (a general methods course and a methods course in English, 
the randomly selected pathway) has an objective related to using assessment data to plan instruction. 

■ There is one lecture on “Lesson Planning — Backwards Design.” 

■ There are no required textbooks or readings that address any aspect of assessment. 

■ There is no practice (thus no practice aligned with field work) , nor is there collaborative practice on using 
assessment data to plan instruction. 

Sample secondary program earning a “1”: There is very limited instruction or practice that prepares teachers to 
use assessment data to drive secondary instruction. 

m Neither of the two assessment-relevant courses (a general methods course and a methods course in social 
studies, the randomly selected pathway) has an objective related to using assessment data to plan instruction. 

■ There are no lectures on using assessment data to plan instruction. 

■ There are no required textbooks or readings that address any aspect of assessment. 

■ The only practice is aligned with field work and requires preparation of formative assessments. There is no 
collaborative practice. 
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Sample secondary program earning a “2”: Instruction and practice on using assessment data to drive instruction 
is limited but includes coursework that is subject-specific. 

m Of the two assessment-relevant courses (one focused on assessment and a methods course in science, the randomly 
selected pathway), the assessment-focused course has the one objective related to using assessment data to 
plan instruction. 

■ There is one lecture on formative assessment and one lecture on integrating standards with the “understanding 
by design” approach to instructional planning. 

■ A required textbook addresses “understanding by design.” 

■ The only practice is the development of a teaching unit using an “understanding by design” template. (This 
may be done in groups of three.) 

Sample secondary program earning a “3”: Instruction and practice on using assessment data to drive instruction 
is evident and includes coursework that is subject-specific. 

■ The methods course in English, the randomly selected pathway, is the only assessment-relevant course. 

■ Lectures address “Backwards Design: how to use the standards and outcomes to direct your instruction,” 
“Using Backwards Design” and analyzing student writing samples to “plan instruction based on this analysis.” 

■ There are no required textbooks or readings that address any aspect of assessment. 

■ Practice includes a “Pre-Assessment Activity for Shakespeare,” involving the use of formative assessments, 
and practice aligned with field work requiring teacher candidates to “understand the purposes of different 
assessments (including state mandated exams) and how to make instructional decisions that improve student 
performance.” There is no collaborative practice. 

Sample secondary program earning a “4”: Instruction and practice on using assessment data to drive instruction 
is extensive and includes coursework that is subject-specific. 

m Four courses (one on general methods and three on English/language arts methods) address assessment. 

■ Three lectures in an English methods course address use of assessment data for instructional planning. 

■ There are no required textbooks or readings that address any aspect of assessment. 

■ Practice includes a heavily weighted class assignment involving analyzing artifacts of student writing to “assess 
student learning, to assess your instructional practice, and to revise and plan for future writing instruction; ” the 
assignment is presented to class “colleagues.” A separate capstone project requires that teacher candidates supply 
artifacts that demonstrate that they align assessments with content goals and plan backwards from summative 
assessments, while using formative assessments to guide instruction. 
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Appendix 3: Research looking at how assessment is taught in 

teacher preparation programs and the impact on teachers 

One of the only available sources of information about the nature of assessment coursework in initial certification 
programs is found in a cursory survey of 60 institutions conducted in 2010 by two graduate students at George 
Washington University. 50 Teacher educators were asked whether they teach students how to use data, and, if so, 
whether the coursework 1) entailed a stand-alone course or was a component of several courses and 2) was required. 
Of the 25 institutions that responded, almost all (23) indicated that they offered some assessment coursework, and 1 8 
labeled the coursework as required. Respondents were about evenly split on the mode of delivery, with some relying 
on a single course and others relying on multiple courses. 

A second source is a 20 1 1 study conducted by the Educational Testing Service (ETS) on coverage of formative assessment 
by 22 teacher preparation programs in New Jersey. The study found only three programs in which formative assessment 
was specifically mentioned. 51 

The following table categorizes the 26 studies we have located that address the impact on teachers of assessment 
coursework. 

“Studies with stronger design” use some sort of control or comparison group in an experiment, natural or otherwise, 
or use a multiple regression for correlation. These studies have a sample size of 100 or more. 

“Studies with weaker design” have no comparison or control, are often simply case studies with potential selection 
bias, and rely on survey or otherwise qualitative data. These studies have a sample size of fewer than 100. 


Total number of studies 

Studies with stronger design 

Studies with weaker design 


No consideration of 
teacher effectiveness 

Consideration of 
teacher effectiveness 

No consideration of 
teacher effectiveness 

Consideration of 
teacher effectiveness** 

26 

2 

(g).(x) 

0 

16 

(a),(b),(c),(d),(e),(f), 

(h),(j),(k),(l*),(o),(r), 

(s),(v),(w*).(y),(aa*) 

8 

(i*),(m*),(n),(p*),(q), 

(t*),(u*),(z*) 


* Research conducted on an in-service sample with applicability to pre-service preparation. 

** This consideration of teacher effectiveness does not include use of data from standardized student assessments. 


Citations for articles categorized in the table are listed below. These articles were located in the Education Research 
Complete and ERIC (Education Resource Information Center) databases from listings of reports on research conducted 
in the United States, Canada, England and Australia that were published between 2000 and 2011 in peer-reviewed 
journals. 


50 Mann, B., & Simon, T. (July 2010). Teaching teachers to use data. Washington, DC:The George Washington University. 

5 1 http:/ /blogs.edweek.org/ edweek/teacherbeat/20 1 1 / 1 2/how_do_we_train_teachers_to_us.html 
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Do state regulations speak 
to teacher preparation in 
assessment? 


State 


Alabama 

□ 

Alaska 

■ 

Arizona 

■ 

Arkansas 

■ 

California 

■1 

Colorado 

■ 

Connecticut 

■ 

Delaware 

□ 

District of Columbia 

□ 

Florida 

■* 

Georgia 

■ 

Hawaii 

■ 

Idaho 

■ 

Illinois 

■ 

Indiana 

□ 

Iowa 

■ 

Kansas 

■ 

Kentucky 

□ 

Louisiana 

■ 

Maine 

■ 

Maryland 

□ 

Massachusetts 

■ 

Michigan 

□ 

Minnesota 

■ 

Mississippi 

□ 

Missouri 

■ 

Montana 

□ 

Nebraska 

□ 

Nevada 

□ 

New Hampshire 

m** 

New Jersey 

— 

New Mexico 

□ 

New York 

□ 

North Carolina 

■ 

North Dakota 

■ 

Ohio 

■ 

Oklahoma 

■ 

Oregon 

□ 

Pennsylvania 

□ 

Rhode Island 

■ 

South Carolina 

□ 

South Dakota 

■ 

Tennessee 

■ 

Texas 

□ 

Utah 

□ 

Vermont 

□ 

Virginia 

■ 

Washington 

■ 

West Virginia 

■ 

Wisconsin 

□ 

Wyoming 

□ 


Appendix 4: State regulations, accreditation 
standards, professional standards 

State regulations 

All teacher preparation programs, whether housed in public or in private institutions, must 
be approved by the state. The table below summarizes the nature of state regulations with 
regard to whether states explicitly or implicitly require that programs prepare candidates in 
assessment . 52 

Even in states that have relevant regulations, those regulations can differ considerably in focus 
and scope. Some regulations are fairly brief and general. For example, New York’s regulations 53 
state, in part: 

The program shall provide study that will permit candidates to obtain the following 
pedagogical knowledge, understanding, and skills: 

(vii) formal and informal methods of assessing student learning and the means of analyzing 
ones own teaching practice — and skill in using information gathered through assessment 
and analysis to plan or modify instruction, and skill in using various resources to enhance 
teaching. 

In contrast, Louisiana’s regulations 54 are extensive and specific: 

Component III.D. The teacher demonstrates ability to assess and facilitate student 
academic growth 

Attributes: III.D. 1 Consistently monitors ongoing performance of students. 

The beginning teacher will 1) use a variety of assessment methods, including technology, that 
are appropriate for evaluating student achievement and instructional goals and objectives; 
2) communicate assessment criteria and standards to students; 3) adjust instruction based 
on ongoing assessments of student understanding; and 4) analyze assessment results to help 
plan instruction for groups of students or individuals. 

III.D. 2 Uses appropriate and effective assessment techniques. 

The beginning teacher will 1) use curriculum-embedded and standardized 
assessment to assess progress; 2 ) design assessments, where appropriate, that reflect 
real-world applications of knowledge and understanding; 3) promote students’ 
use of self- monitoring and self-assessment; and 4) use alternative instructional 
approaches and assessments to ensure that all students learn and succeed. 


52 States can implicitly require such preparation by specifying professional competencies for teachers 
that include knowledge of assessment. 

53 Regulations of the Commissioner of Education 52.21. (Access at: http://www.highered.nysed. 
gov/ocue/52.21.htm) 

54 Guidelines for the Submission and Review of Redesigned teacher Preparation Programs, “Components 
of Effective Teaching.” (Access at: http://www.laregentsarchive.com/Academic/TE/redesign_ 
guidelines.pdf) 


* Only with regard to reading instruction 

** Only with regard to elementary 
preparation programs 
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III.D.3 Provides timely feedback to students regarding their progress. 

The beginning teacher will 1) use appropriate language and formats to provide each student with timely 
feedback that is accurate, constructive, substantive, and specific; 2) promote students’ ability to use feedback 
to guide and enhance their learning; and 3) base feedback on high expectations for student learning. 
III.D.4 Produces evidence of student academic growth under his/her instruction. 

The beginning teacher will 1 ) collect and maintain accurate records; 2 ) analyze and interpret assessment 
data; and 3) summarize assessment results to share with students, families, and school administrators. 

Moreover, additional preparation requirements related to assessment may not be evident in regulations. For example, 
Oregon does not appear in our review of regulations to have any regulations that touch on the preparation of teachers 
in assessment, but candidates seeking initial licensure must submit to the state two “work samples” that include 
assessments, ss analysis of assessment data and the use of assessment data in instructional plans. 56 

The difficulty of ascertaining the exact nature of requirements for teacher preparation (especially when survey questions 
concern both teacher preparation requirements and requirements for preparation of administrators) may explain 
inconsistent responses from state personnel in several surveys conducted by the DQC. In a broad survey about many 
aspects of state initiatives on use of data, the DQC asked state agency personnel about whether training on data driven 
instruction figured into their state’s approval process for educator and/or leader preparation programs: 

Does your state ’s program approval process re q uire educator and/or leader preparation programs to demonstrate 
that they are adequately training their candidates to analyze, interpret, and use student- and aggregate-level data 
to adapt classroom, building, and district practices based on student need? 

About two dozen states answered affirmatively in each of the two most recent surveys, but a large number of states 
answered inconsistently, with their answer in one survey not matching their answer in the second, including states 
whose affirmative answer in the earlier survey was not repeated in the subsequent survey. 

Institutional accreditation standards 

NCATE is the largest accreditor of teacher preparation programs, having accredited about half of the nation’s 1,400 
institutions offering such programs. 

NCATE has a standard for what teacher candidates should learn about assessment. 57 Expectations are fairly low. Only 
candidates in advanced certification programs (not candidates in initial certification programs at the undergraduate or 


55 It does require that administrators “demonstrate the ability to use aggregated and disaggregated student achievement data to 
develop effective instructional programs.” 

56 Oregon’s requirements (Administrative Code 584-017-0185) can be accessed at http://arcweb.sos.state.or.us/pages/rules/ 
oars_500/oar_584/584_0 1 7.html. 

Other states requiring teacher work samples that address assessment explicitly include Washington (WAC 181-78A-255), 
Kansas (see the Department of Education’s Kansas Performance Teaching Portfolio: Content Guidelines, 201 1, at http:// 
www.ksde.org/LinkClick.aspx?fileticket=_FxbAFAEop4%3D&tabid=3769&mid=l 1692) and Missouri (see the Department of 
Elementary and Secondary Education’s Missouri Standards for Teacher Preparation Programs and Benchmarks for Preliminary 
Teacher Education Programs at http://www.dese.mo.gov/schoollaw/rulesregs/documents/MoSTEP_10-06.pdf 

57 NCATE also published a report in 20 1 0 on assessment in teacher preparation: Assessment as a Critical Element in Clinical 
Experience for Teacher Preparation, which can be accessed at http:/ /www.ncate.org/LinkClick.aspx?fileticket=oo50CSYDEF 
M%3D&tabid= 715 
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graduate level) are expected to have a “thorough knowledge” of assessment that includes making “data-driven decisions.” 58 

TARGET 

Teacher candidates focus on student learning and study the effects of their work. They assess and analyze student 
learning, make appropriate adjustments to instruction, monitor student learning, and have a positive effect on 
learning for all students. Candidates in advanced programs for teachers have a thorough understanding of assessment. 
They analyze student, classroom, and school performance data and make data-driven decisions about strategies 
for teaching and learning so that all students learn. They collaborate with other professionals to identify and design 
strategies and interventions that support student learning. 

Professional standards 

Many states have professional standards for teachers that guide the nature of preparation provided to teacher candidates. 
These standards often articulate the professional competencies that coursework and clinical practice are designed to 
build. Thirty-six states 59 have adopted as their professional standards the model core teaching standards issued by the 
Interstate Teacher Assessment and Support Consortium (InTASC). 60 In 20 1 1 , InTASC issued a revised assessment standard: 

The teacher understands and uses multiple methods of assessment to engage learners in their own growth, to 
monitor learner progress, and to guide the teacher’s and learner’s decision making. 

(The standard can be found in its entirety below.) 

The substantive differences between this new standard and the previous InTASC assessment standard are few: “Formative” 
and “summative” assessments are referenced in the new standard, whereas the old standard referenced “informal” and 
“formal” assessments used for the same instructional purposes; collaborative work to examine test and other performance 
data is included in the new standard, whereas only individual work was mentioned in the old. 

Of some note is a new performance goal also added to the InTASC collaboration standard: 

The teacher participates actively as part of an instructional team, giving and receiving feedback on practice, examining 
student work, analyzing data from multiple sources, and sharing responsibility for each student’s learning. 


58 Access at: http:/ /www.ncate.org/Standards/NCATEUnitStandards/UnitStandardsinEffect2008/tabid/476/Default. 
aspx#stndl 

59 Although the consortium has always emphasized that state professional standards should only use InTASC standards as a 
foundation, most states that are members use the standards verbatim. 

60 Access at: http:/ /www.ccsso.org/Documents/20 1 1 /InTASC_Model_Core_Teaching_Standards_20 1 1 .pdf 
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InTASC Standard #6 - Assessment The teacher understands and uses multiple methods of assessment to 
engage learners in their own growth, to monitor learner progress and to guide the teacher's and the learner's 
decision making. 

Performances 

6(a) The teacher balances the use of formative and summative assessment as appropriate to support, verify and document 
learning. 

6(b) The teacher designs assessments that match learning objectives with assessment methods and minimizes sources 
of bias that can distort assessment results. 

6(c) The teacher works independently and collaboratively to examine test and other performance data to understand 
each learner's progress and to guide planning. 

6(d) The teacher engages learners in understanding and identifying quality work and provides them with effective descriptive 
feedback to guide their progress toward that work. 

6(e) The teacher engages learners in multiple ways of demonstrating knowledge and skill as part of the assessment process. 

6(f) The teacher models and structures processes that guide learners in examining their own thinking and learning as 
well as the performance of others. 

6(g) The teacher effectively uses multiple and appropriate types of assessment data to identify each student's learning 
needs and to develop differentiated learning experiences. 

6(h) The teacher prepares all learners for the demands of particular assessment formats and makes appropriate accommodations 
in assessments or testing conditions, especially for learners with disabilities and language learning needs. 

6(i) The teacher continually seeks appropriate ways to employ technology to support assessment practice both to engage 
learners more fully and to assess and address learner needs. Critical 

6(o) The teacher knows when and how to evaluate and report learner progress against standards. 

6(p) The teacher understands how to prepare learners for assessments and how to make accommodations in 
assessments and testing conditions, especially for learners with disabilities and language learning needs. 

Essential Knowledge 

6(j) The teacher understands the differences between formative and summative applications of assessment and knows 
how and when to use each. 

6(k) The teacher understands the range of types and multiple purposes of assessment and how to design, adapt or select 
appropriate assessments to address specific learning goals and individual differences and to minimize sources of bias. 

6(1) The teacher knows how to analyze assessment data to understand patterns and gaps in learning, to guide planning 
and instruction and to provide meaningful feedback to all learners. 

6(m)The teacher knows when and how to engage learners in analyzing their own assessment results and in helping to 
set goals for their own learning. 

6(n) The teacher understands the positive impact of effective descriptive feedback for learners and knows a variety of 
strategies for communicating this feedback. 

6(o) The teacher knows when and how to evaluate and report learner progress against standards. 

6(p) The teacher understands how to prepare learners for assessments and how to make accommodations in 
assessments and testing conditions, especially for learners with disabilities and language learning needs. 
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Dispositions 

6(q) The teacher is committed to engaging learners actively in assessment processes and to developing each learner's 
capacity to review and communicate about their own progress and learning. 

6(r) The teacher takes responsibility for aligning instruction and assessment with learning goals. 

6(s) The teacher is committed to providing timely and effective descriptive feedback to learners on their progress. 

6(t) The teacher is committed to using multiple types of assessment processes to support, verify and document learning. 

6(u) The teacher is committed to making accommodations in assessments and testing conditions, especially for learners 
with disabilities and language learning needs. 

6(v) The teacher is committed to the ethical use of various assessments and assessment data to identify learner 
strengths and needs to promote learner growth. 
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Appendix 5: A short tutorial on the taxonomy of assessment 

While “norm-referenced” student performance data (provided by tests such as the SATs) that compare a student’s 
performance to the performance of peers nationwide are used by school districts, the classroom teacher is most concerned 
with data from “criterion-referenced” assessment designed to determine whether a student has sufficiently mastered 
specific academic content defined by the state’s K-12 learning standards. 61 

In turn, criterion-based assessment falls into two broad categories: internally and externally validated assessments. 
Internally validated, or “classroom assessments,” are possibly developed, but at least selected, by the classroom teacher 
or a team of teachers at a school. These assessments are administered on a schedule determined by the teachers and 
scored by a scale at the teachers’ discretion. Externally validated assessments are interim or final “standardized” assessments 
whose content and scoring are uniform across a district or a state. These assessments are administered on a schedule 
determined by districts or state agencies. 62 

Classroom assessments can be further divided into categories based on type (alternative, authentic or performance, as 
opposed to traditional “pencil and paper”) and purpose (“formative assessment” informs teacher instruction whereas 
“summative assessment” informs student evaluation). 63 


Type of classroom assessment 



*Also known as “alternative" or “performance-based.” 


The categorization of student performance data is more fluid than this categorization suggests. For example, most standardized 
assessments are designed to serve as end-of-year assessments of mastery and are therefore summative, but they can also 
provide rich data for guiding instruction in the succeeding school year, thus taking on a formative purpose. 


61 Another category of assessment is termed “ipsative” — comparing a student’s current performance to past performance. If 
standardized assessments are vertically scaled, they can serve both a summative and an ipsative purpose. 

62 It follows then that classroom assessment could also be broadly defined as “internal assessment” and standardized assessment 
broadly defined as “external assessment.” 

63 Even this categorization does not exhaust other possible ones. For example, assessment is either “objective” (a classroom 
or standardized assessment on which each question has only one correct answer) or “subjective” (an assessment on which 
each question can be correctly answered in more than one way) . Also useful is the categorization of classroom or standardized 
assessment as “formal” (i.e., using a written document and producing a numeric score or “informal” (i.e., less structured in 
format and more casually scored) . 
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This report is available online at 

www.nctq.org/p/publications/docs/assessment_report.pdf 
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